Localizing facial landmarks can be an important component in many computer vision applications such as face recognition, facial expression recognition, face reconstruction, and re-targeting. Many approaches have been proposed with varying degree of success on benchmark datasets composed by mostly frontal images in controlled setting. However, accurate localization of facial landmarks in real-world, cluttered images is still a challenge due to a number of factors. FIGS. 1A-1E show examples of facial images that create facial recognition challenges. FIG. 1A shows an example of a facial image in a non-ideal pose. The face is looking down relative to the camera. FIG. 1B shows an example of a facial image with unusual expression. FIG. 1C shows an example facial image with illumination varying across the face. The left side of the face is well illuminated and the right side of the face is in shadow. FIG. 1D shows an example of a facial image with a portion of the face occluded. In this image the left eye is occluded by a hat. FIG. 1E shows an example of a low quality image. In this example, the face is out of focus. Each of these examples, among others, pose challenges to facial recognition techniques.